REMARKS 

Reconsideration and further examination is respectfully requested. 

The Applicant notes that the Examiner has rejected claims 1-3 and 14-15 based on Patent 
Application No. US 2002/0118682 (Choe et al.) under 35 U.S.C. § 102(e). Withdrawal of the 
rejection is respectfully requested. 

Choe et al. lists a U.S. filing date of December 20, 2001. This date, however, is after the 
filing date of the present application, which was filed on November 9, 2001. Thus, the effective 
filing date of Choe et al. must be either the foreign application priority date of February 15, 2001, 
based on a Korean patent application, or the filing date of U.S. Provisional Application filed on 
December 22, 2000. 

The foreign priority date does not qualify as an effective filing date for purposes of prior 
art. Under 35 U.S.C. §102(e), in order to rely on the foreign priority date the foreign application 
must be a WIPO publication under PCT article 21(2), be based on a PCT application designating 
the United States, and published in English. MPEP § 706.02(f)(1). The foreign application in 
question is not a PCT International application, and therefore cannot claim the benefit of any 
early filing date as a prior art reference. 

Accordingly, the effective filing date of the reference is the provisional filing date of 
December 22, 2000. In this regard, the MPEP states "[i]f the application properly claims benefit 
under 35 U.S.C. 119(e) to a provisional application the effective filing date is the filing date of 
the provisional application for claims which are fully supported under the first paragraph of 35 
U.S.C. 1 12 by the provisional application. MPEP § 706.02(V)(D). 

Any rejection of claims of the present application based on Choe et al. must be based on 
disclosure contained in the provisional application to which the published patent application 



claims priority. The provisional application is enclosed herewith, and a review of this reference 
shows that the rejection of claims 1-3 and 14-15 cannot be supported based on this disclosure. 
Portions of the published patent application relied on by the Examiner do not appear in the 
provisional application. 

The Choe et al. provisional does not teach or suggest the use of an LC-trie algorithm for 
use in connection with an IP-address lookup table. In fact, the Choe et al. provisional actually 
teaches away from such an approach. Section 2 of the Choe et al. provisional discuss previous 
work, and begins with a review of existing router architecture. Then the reference discusses 
difficulties encountered in the prior work when it came to adding new routes and deleting old 
routes in IP-address lookup tables (page 5). Next, ensues a discussion of various data structures, 
including one based on the work of Nillson and Karlsoson (page 6, footnote 17). This is in fact 
the very work upon which the present application is based. Rather than adapting the LC-trie 
approach proposed in abstract by Nillson and Karlsoson, the Choe et al. provisional teaches 
against its use (page 6). In specific reference to the work of Nillson and Karlsoson, Choe et al. 
states "[although previously mentioned schemes contributed to give the reduction of lookup 
time, there still exists a problem concerning to [sic] the route updates." (page 6). Choe et al. 
then goes on to expressly abandon these prior approaches in favor of a different approach 
described as a randomized algorithm (page 6). 

The Choe et al. provisional makes reference, and gives consideration, to the very same 
LC-trie compression model adapted and claimed in the present application, and rejects its use for 
the purpose of maintaining routing and forwarding tables of IP-addresses. Thus, the Choe et al. 
provisional patent application actually teaches not to use the LC-trie approach in combination 
with the forwarding and routing tables. 



The approach favored by Choe et al. is actually quite opposite to that of the present 
invention. Choe et al. states "[t]he inverted key value sequence means that the search will be 
started from leaves to root unlike to [sic] the traverse sequence in a trie or a tree data structure" 
(page 7). 

The Choe et al. provisional does not teach each element of the claimed invention, but 
merely recites the prior art and then teaches an invention that substitutes a completely different 
algorithm for IP-address lookup despite being aware of the underlying algorithm upon with the 
present invention is based. Accordingly, Applicant respectfully submits that the cited prior art 
does not establish a prima facie case of novelty under 35 U.S.C. § 102(e) as applied to claims 1-3 
and 14-15, and therefore requests withdrawal of the refusal of these claims. 

Applicants have made a diligent effort to place the claims in condition for allowance. 
However, should there remain unresolved issues that require adverse action, it is respectfully 
requested that the Examiner telephone Daniel A. Rosenberg, Applicants' Attorney at 515-288- 
2500 so that such issues may be resolved as expeditiously as possible. 

For these reasons, and in view of the above amendments, this application is now 
considered to be in condition for allowance and such action is earnestly solicited. 
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High-speed IP routing lookups and routing table 

management 

Myongsu Choe 
Samsung Electronics Company LTD. 
Sungnam-Shi, Kyunggi-Do, S. Korea 
Email : mschoe@samsung.co.kr 

Abstract 

IP route lookup and its table management have been considered as one of the 
major bottlenecks in the design of a high-speed router. We present, in this paper, a 
randomized algorithm which proves to be efficient in the lookups and management 
of an IP routing table (or forwarding table) in comparisons with other trie-based 
algorithms. The algorithm is intended to use in a distributed or a parallel router 
architecture where a forwarding engine on each line card module or a forwarding en- 
gine separately located from a line card module is processing incoming packets apart 
from a routing processor in which the routing table is computed and maintained. 

We compare complexity of our proposed algorithm to other ones. Our analyt- 
ical results indicate that a variant of the randomized algorithm, a skip list with 
fc-ary hashed nodes, significantly reduces the number of memory accesses required 
to update route changes and find a route with the longest prefix matching from the 
routing table (or forwarding table) as well as the amount of memory space to store 
route entries. 

Key words: IP route lookups, longest prefix matching, forwarding engine, ran- 
domized algorithm. 

1 Introduction 

According to the increasing number of Internet users, variety of supported services, and 
the expansion of service areas such as VoIP and stream-oriented applications, the traffic 
in the Internet has been increased exponentially. Forwarding packets to the next hop 
interface by finding a destination path without causing any delay in a high-speed router 
has been emerged as a contingent design issue. In order to find the destination path, 



managing a routing (or forwarding) table in a compact form and reducing lookup time are 
required. 

In a traditional router before the advent of recently deployed high-speed routers where 
a required time to process packets and find their destinations is faster than the one on the 
transmission paths, a router as a relaying node connects subnetworks or other networks 
has played its roles well. Recently, the bandwidth increase of optical networking interface 
such as EP over DWDM surpassed processing time in a router and raised a blame that a 
router causes a dominant bottleneck in a high-speed Internet [1, 2, 3, 4, 5]. 

In addition, an introduction of a new IP addressing scheme called CIDR (Classless Inter- 
Domain Routing) [7, 8] to use IP addresses efficiently to prevent its address space from 
depleting proposed a different matching scheme called an LPM (Longest Prefix Matching) 
instead of an exact matching. Several algorithms usually using trie (or Patricia trie) has 
been widely used. This paper propose a longest prefix matching algorithm based on a 
randomized approach in which it will be used in the case of IPv6 deployment. 

In this paper, we introduce a variant of the randomized algorithm which can provide 
minimal lookup time and an efficient maintenance of up-to-date routing table by reflecting 
routes changed from the routing processor due to the route flaps from neighboring routers. 
Furthermore, our scheme is planned to apply an 80-320 Gbps DP Switch which is developing 
by Samsung Electronics Company. 

The remainder of this paper is organized along the following lines. Section 2 contains 
a description of our proposed high-speed router architecture as well as the previous route 
. lookup work, section 3 describes our proposed algorithm, section 4 compares complexity 
of our algorithm and other typical ones, and the final section contains our conclusions. 

2 Previous work 

The recently announced or deployed backbone routers have been going on a distributed 
architecture as shown in Figure 1 to provide efficient packet processing in comparison with 
a centralized one where all the incoming packets from line cards are only processed in a 
routing processor. 

A router consists of line caid modules with a forwarding engine respectively, a routing 
processor and a switch fabric. For reliable operations, each different card can have its own 
redundant one as a standby purpose. Packets coming from other neighboring router(s) 
pass through the line card to switch fabric. The routing processor initially builds up its 
routing table and maintains it by reflecting changed routes. The switch fabric switches 
packets from input line cards to output ones and vice versa, i.e., relaying packets between 
input and output line cards. 



RT: Routing Table 



UnecarimoduMFE 1 Line card module wfoFE 




FT: Forwarding Table 



Figure 1: Distributed router architecture 

Chan et al. [3] classifies two major lines of high-speed router's architecture as distributed 
or parallel one where its major distinction between them is dependent upon the location of 
the forwarding engine. As shown in Figure 1, the forwarding engine is embedded in a line 
card in which it connects other network node via a network interface. Almost commercial 
high-speed routers such as Cisco's Catalyst 8510 and PowerRail 5200 from Packet Engines, 
Inc., GigaRouter from NetStar including Samsung's Galaxy have the similar architecture 
as shown in Figure 1. On the other hand, Some experimental models such as Bell Labs 
[6] and BBN [4] followed on the track of the parallel architecture in Figure 2. 

As the forwarding engine is separated from the line card, the parallel architecture uses 
a client and server based model for handling route lookups, parallel packet processing 
in each forwarding engine. In addition, there is a possibility to provide scalable packet 
processing capability as implied to mathematical analysis by Chan et al. [3]. Samsung 
also constructed a combined scheme in Figure 3 in which it integrates the parallel archi- 
tecture with the distributed one, thereby providing cost-effective solutions by assigning a 
forwarding engine in a a line card with high-speed data link and by sharing a forwarding 
engine into a few line cards with low-speed data links. 

The recently updated routing table generated from routing protocols such as RIP, OSPF 
or BGP-4 exists in the routing processor. A compact table called a forwarding one de- 
signed for an efficient lookup is copied from the routing table in a routing processor. The 
forwarding table is only designed for an efficient lookup by sacrificing the efficiency of 
route additions and deletions. 
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Figure 2: Parallel router architecture 
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Figure 3: A combined scheme of Distributed and Parallel router architecture 

If an incoming packet from an input link cannot find its destination path from its 
forwarding table, the corresponding packet must pass through the switch fabric to the 
routing processor to resolve its unmatched route. After finding its destination, then the 
packet must route to the switch again to transfer the packet to the output line card module. 
Otherwise, the packet is silently discarded at the routing processor due to the non-existing 
destination. The details of a forwarding engine and a routing processor showing the packet 
flow are also depicted in Figure 4 and 5. 

The separation of routing table and forwarding one has been designed for the purpose 
of high speed packet processing where the routing table must be maintained to reflect 
the recent route changes. On the other hand, the forwarding table has been focused to 
support efficient lookup operation and to keep the whole forwarding table within its cache 
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Figure 4: Structure of a forwarding engine 
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Figure 5: Structure of a routing processor 



as compact as possible. Although the forwarding table (or routing one) is designed for an 
efficient LPM matching and the compactness of the table, adding new routes or deleting 
obsolete ones is inefficient. 

Srinivasan and Varghese [11] showed interesting results by analyzing empirical data 
collected by IPMA [24]. Added or canceled routes in a day by a BGP routing protocol at 
a transit AS domain amounts to be 1,899,851 from the data recorded on Nov. 1, 1997. It 
means that a backbone router should perform 25 prefix updates/sec. We infer from the 
data that a data structure of a routing (or forwarding) table must be carefully designed 
to update route change promptly. 

Typically used data structures in a routing table such as a radix tree or a Patricia 
trie caused increasing number of memory accesses to find a neighbor interface in order to 
transfer a packet to destination. Degermark et at [10] proposed a compact forwarding 



table capable to store in a cache of the forwarding engine except it is hard to reflect 
changed routes into it. A rope search algorithm [14, 15] by mapping a trie structure to a 
binary tree with hash tables and a complete prefix trie based on multi-resolution trie by 
Tzeng et <d. [13] tend to be inefficient due to the update of changed route entries because 
those data structures are based on the trie. Aside from those ones, other variants of trie 
have been proposed. For example, a two-trie scheme [16] by linking two tries to reduce 
search time and an LOtrie [17] to reduce level length in a trie, and a DP-trie [9] were 
suggested. 

Although previously mentioned schemes contributed to give the reduction of lookup 
time, there still exists a problem concerning to the route updates. Hardware-assisted 
schemes are also suggested to reduce lookup time. Gupta et al [19] proposed a solution 
based on the use of large scale memory. Reducing lookup time is possible in comparisons 
with software-based ones, but it still poses significant amount of memory use and cost 
in the transition of IPv6. A scheme using CAM (Content Addressable Memory) was 
suggested by McAuley et d. [20], but due to the high price of CAM, it is not considering 
to use it at present. Huang et al. [21] proposes an indirect lookup algorithm using a 
pipelined memory access to reduce memory accesses although it has a disadvantage over 
IPv6 transition. 

In this paper, we suggest an algorithm based on a randomized approach in which it 
will allow minimal memory accesses and easy maintenance of a routing or a forwarding 
table. In the case of route entry maintenance, it will reflect only changed route(s) instead of 
constructing the whole table. In addition, some algorithms [10, 11, 12, 15] demanded routes 
of prefix to be sorted prior to the table construction. On the other hand, our algorithm 
can be used to construct the corresponding table without any sorting in advance. 

3 Randomized algorithm 

A randomized algorithm is one that makes random choices during its execution. The 
behavior of such an algorithm may be random even on a fixed input. It focuses on estab- 
lishing that it is likely to behave well on every input. The advantages of its use come from 
its simplicity and efficiency. We propose a variant of a skip list proposed by Pugh [22] for 
supporting efficient lookup and managing a routing table with ease in a high-speed router. 

A skip list is considered as a linked list with sorted n nodes and is usually used in the 
data structure on behalf of a balanced tree such as splay tree [23]. Thus it is simpler 
and more efficient than the balanced tree for insert and delete operations. We propose a 
variant of skip list for the longest prefix matching as shown in Figure 6. 

Constructing a routing table or inserting changed routes into it can be performed with- 
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Figure 6: A variant of skip list with A;-ary hashed nodes 

out any distinctions. Each node has a single or multiple pointers and we name the leftmost 
node as a header node because all the operations on the variant of skip list such as lookup, 
insertions, deletions, and updates must be started from the node. 

As shown in Figure 6, the header node contains +oo as a key value and a pointer to 
indicate next node. A key in each node means prefix length range or clustering of prefix 
length value and the key is inserted in a descending order. The inverted key value sequence 
means that the search will be started from leaves to root unlike to the traverse sequence 
in a trie or a tree data structure. The key value in the first node except the header one 
has the corresponding prefix range "32-24," and the route entry falling between the prefix 
length range is stored at one of the hash tables matching to its prefix length. 

In Figure 6, the prefix length range is divided into a fixed way but variable length 
division scheme also expects to be possible. Srinivasan and Varghese [11] used a dynamic 
programming technique to divide the prefix range to provide optimal storage. In our 
proposed algorithm, we use fixed prefix range division. For the case of IPv4, if the prefix 
length range is fixed to 8, then the total number of inserted nodes can be 5 * [(1 + 32/8)] . 
As shown from the empirical result from [24], the distribution of actual referenced prefix 
length has a skewed one, i.e., implying the locality pattern existing in an actual referenced 
prefix length distribution. Therefore, we may expect to use smaller number of nodes. 

A variable M axLevel represents the maximum level number among all the nodes be- 
longing to the skip list, t.e., the total number of pointers at the header node to point other 
nodes. Each node keeps a key value key(x) and pointer(s) next(x, L) to denote a pointer 
from the node x to point a neighboring node at the level L of node x. 



We let each key in a node have prefix length range. In Figure 6, the first node except 
the header one has route entries ranging from 32 to 24 and each prefix length has its own 
hash table to store route entries matching to the corresponding prefix. If there exist all the 
route entries with prefix ranging from 32 to 24, then the corresponding node has 9 pointers 
to point 9 different hash tables. Each hash table only stores route entries matching its 
prefix length exactly. That is why we call a variant of skip list as a skip list with A;-ary 
hashed node(s). 

We represent insert, lookup, delete, and update operations respectively as pseudo code 
in Figure 7,8,9,10. 

4 Complexity analysis 

Actually, it is not easy to compare all of the algorithms on the same platform because 
some of the algorithms were experimented in various computing environments. Therefore, 
it is desirable to compare our algorithms with other ones in terms of complexity [25]. 

Assume that N route entries exist in a routing table. To represent an IP address, W 
address bits are required and A: is defined as an algorithm dependent constant. Table 1 
shows compared complexity of our algorithm and other ones in terms of time and memory 
space. 
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Complete prefix tree 


0{NlgW) 






0{W/h) 


o(if) 


Binary hash table searcb 


0{NlgW) 






0{lg{W}) 


0{N) 


Multrway and mutticolmnn search 


0{NW) 


O(N) 


0{N) 


0{lg k N) 


0{N) 


Large memoiy architecture 


O(N) 






0[W/k) 


0{N) 


Skip list 


0{NlgN) 


6(lgrf) 


0{lgN) 


O(mgN) 


0(N) 


Ours 


0(Nlg(W/k)) 


0(lg(W/k)) 


0{lg{Wjk)) 


0(lg(WJh)) 


0(N) 



'Tzeng ef d. [13] analyzed its complexity as 0(lg{N) + W). On the other hand, Waldovogel et al 
[14] showed it as 0(lg{2N)). 



To avoid confusion from other logarithms with a base e or 10, Ig denotes a logarithm 
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var k: interger init 0; (* prefix length *) 
prefix: bitstring init V0'; 
interface: integer init 0; 

Insert{k,prefix,interface) (* inserting a route entry *) 
begin 

newLevel := RandomLevelQ; 
allocate Nodes[newLevel+l]; 
while L > 0 

if next{x,L) = null or key(next(x,L)) > k then 

if newLevel > L then Nodes[L] := x; 

L:=W; 

else 

x := next(x,L); 
end if 
end while 

if key{x) f k then (* not found *) 
n := MafceNode(k,newLevel); 
for i := 0 to newLevel 

next(n,i) : = next(Nodes[i],i); 
next(Nodes[i],i) : = n; 
end for 
end if 

hash-insert (prefix^interface) 

end 

RandomLevelQ (* deciding next level randomly *) 
begin 
L:=0; 

r := Random (); (*r€[0,l] *) 
while r < p and L < MaxLevel 

L : = L+l; 
end while 
return L; 

end 



Figure 7: Insert operation 
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Searchikjprefix) (* looking for a route with the longest prefix *) 
begin 

x := Header; 
L := MaxLevel; 
while key{x) > k and L < 0 
if next(x,L) = null then 
L := L-l; 

else 

if key{next(x,L)) < k then 

L := L-l; 
else 

x := next(xJL); 
end if 
end if 
end while 

if L < 0 or key (x) £ k then 
return null; 

else 

return hash-get(prefix); 
end if 



Figure 8: Lookup operation 



Deleteikjprefix.interface) (* deleting a route entry *) 



if search (k, prefix) then 

hash-delete (prefix, interface); 

else 

report error; 
end if 

end 



Figure 9: Delete operation 
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Update(k,prefix,interface) (* updating a route entry *) 
begin 

if search (k, prefix) then 

hash-update (prefix, interface); 

eke 

report error; 
end if 

end 

Figure 10: Update operation 

with base 2. In the case of IPv4, W corresponds 32 and 128 for IPv6. The MAE-EAST 
known as routers in a NAP (Network Access Point) has around 50,000 entries. In this 
case, k in a skip list with k-aiy hashed nodes is 4 if prefix range is equal to 8. It amounts 
to be 16 for IPv6 case. 

Our algorithm is better than the traditional search tree and tries (Patricia trie or LC- 
trie) in comparisons with routing table construction, route lookup, insert, and delete 
operations. The complexity results indicates that our proposed algorithm outperforms a 
pure skip list and even binary hash table search [14] and a complete prefix trie [13] known 
to be licensed in commercial routers. 

5 Conclusion 

In this paper we introduced a skip list with A;-axy hashed nodes which can be applicable 
to the route lookup algorithm in a high-speed router. We show that our algorithm outper- 
forms previously proposed ones which only emphasize the efficiency of route entry look-up 
and the compactness of its table rather than table build-up, route lookup and update. 

• We propose an integrated scheme of parallel and distributed router architecture. It 
seems to be a cost effective solution by dedicating a forwarding engine to an inter- 
face with high-speed data link layer and sharing autonomous forwarding engine(s) 
with interfaces with low-speed data links. By using load balancing, it may improve 
router system throughput. We are exploiting this possibility of the architecture by 
mathematical analysis and simulation. 

• We suggest to use routing and forwarding tables with the same data structure for 
easy maintenance. 
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• Our proposed algorithm cau be used in IP routing areas. 

• Our algorithm also can be used in IPv6. 

• Among the main reasons to use a randomized algorithm in the areas of route lookup, 
table construction and its update, the algorithm is known to be operating well on 
every input from probabilistic analysis. Our complexity comparisons with other ones 
indicate that it is better than the other algorithms in the worst case comparisons. 

• We used an evenly fixed division scheme of prefix length range to make reduction of 
nodes in a skip list. It will be worth efforts to exploit variable prefix length range 
(or clustering) division by adapting prefix length distribution with locality pattern 
as implied from IPMA's actual gleaned data. 

• By descending key value in each node of a skip list, the lookup operation starting 
from a header node searches the longest prefix matching from the leaves opposite 
to the case of a trie or a tree. Thus backtracking will not happen and a specially 
designed pointer called a marker [14] is not required, thereby expecting to save 
storage. 

• Due to the characteristics of a skip list, there is no required for sorting in order to 
construct or insert route entries in a routing (or forwarding) table. On the other 
hand, some algorithms [10, 11, 12, 13, 14, 18] needs to be sorted by prefix length 
sequence to build up the table. 

• As a fringe benefit, on the current trend of a contemporary architecture such as 
a distributed router architecture or a parallel one where a forwarding table in a 
forwarding engine still separate from a routing one in a routing processor, by trans- 
ferring only updated routes along the paths into the router instead of the whole 
route entries, memory bandwidth or system bus contention may be relieved. 

• In addition, the reduction of lookup time and the maintenance of the consistent rout- 
ing information over all the routing related tables may contribute to the performance 
enhancement over other factors such as QoS, multicasting, IPSec, and others. 
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Figure 1: Distributed router architecture 
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Figure 2: Parallel router architecture 
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Figure 3: A combined scheme of Distributed and Parallel router architecture 
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Figure 5: Structure of a routing processor 
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Figure 6: A variant of skip list with fc-ary hashed nodes 



var k: interger init 0; (* prefix length *) 
prefix: bitstring init 6'0'; 
interface: integer init 0; 



Insert(k, prefix^ interface) {* inserting a route entry *) 
begin 

newLevel := RandomLevel(); 
allocate Nodes[newLevel+l]; 
while L > 0 

if next(x,L) - null or hey{next{x,L)) > k then 

if newLevel > L then Nodes[L] := x; 

L := L-l; 

else 

x := next(x,L); 
end if 
end while 

if key(x) ^ k then (* not found *) 
n := MakeNode(k,newLevel); 
for i := 0 to newLevel 

next(n,i) : = next(Nodes[ij,i); 
next(Nodes[i]4) : = n; 
end for 
end if 

hash-insert (prefix^nterface) 

end 

RandomLevd() (* deciding next level randomly *) 
begin 
L:= 0; 

r:= Random (); (*r€[0,l] *) 
while r < p and L < MaxLevel 

L:=L+1; 
end while 
return L; 

end 



Figure 7: Insert operation 



Searchfcprefiz) (* looking for a route with the longest prefix *) 
begin 

x := Header; 
L := MaxLevel; 
while key(x) > k and L < 0 
if next(x 7 L) = null then 
L := L-l; 

else 

iffcey(nearf(i,X)) <fcthen 

L := L-l; 
else 

x := next(x,L); 
end if 
end if 
end while 

if L < 0 or key(x) # k then 
return null; 

else 

return hash-get (prefix); 
end if 



Figure 8: Lookup operation 



Deletei^prefix.int&rface) (* deleting a route entry *) 
begin 

if search (k, prefix) then 

hash-delete (prefix, interface); 

else 

report error; 
end if 



Figure 9: Delete operation 



Update{k,prefix, interface) (* updating a route entry *) 
begin 

if search (k, prefix) then 

hash-update (prefix, interface); 

else 

report error, 
end if 



Figure 10: Update operation 



